Enriching remote labs with computer vision and drones

165 p.With the technological advance, new learning technologies are being developed in order to contribute to better learning experience. In particular, remote labs constitute an interesting and a practical way that can motivate nowadays students to learn. The studen can at anytime, and from anywhere, access the remote lab and do his lab-work. Despite many advantages, remote tecnologies in education create a distance between the student and the teacher. Without the presence of a teacher, students can have difficulties, if no appropriate interventions can be taken to help them. In this thesis, we aim to enrich an existing remote electronic lab made for engineering students called "LaboREM" (for remote Laboratory) in two ways: first we enable the student to send high level commands to a mini-drone available in the remote lab facility. The objective is to examine the front panels of electronic measurement instruments, by the camera embedded on the drone. Furthermore, we allow remote student-teacher communication using the drone, in case there is a teacher present in the remote lab facility. Finally, the drone has to go back home when the mission is over to land on a platform for automatic recharge of the batteries. Second, we propose an automatic system that estimates the affective state of the student (frustrated/confused/flow) in order to take appropriate interventions to ensure good learning outcomes. For example, if the studen is having major difficulties we can try to give him hints or to reduce the difficulty level of the lab experiment. We propose to do this by using visual cues (head pose estimation and facil expression analysis). Many evidences on the state of the student can be acquired, however these evidences are incomplete, sometims inaccurate, and do not cover all the aspects of the state of the student alone. This is why we propose to fuse evidences using the theory of Dempster-Shafer that allows the fusion of incomplete evidence

Enriching remote labs with computer vision and drones

165 p.With the technological advance, new learning technologies are being developed in order to contribute to better learning experience. In particular, remote labs constitute an interesting and a practical way that can motivate nowadays students to learn. The studen can at anytime, and from anywhere, access the remote lab and do his lab-work. Despite many advantages, remote tecnologies in education create a distance between the student and the teacher. Without the presence of a teacher, students can have difficulties, if no appropriate interventions can be taken to help them. In this thesis, we aim to enrich an existing remote electronic lab made for engineering students called "LaboREM" (for remote Laboratory) in two ways: first we enable the student to send high level commands to a mini-drone available in the remote lab facility. The objective is to examine the front panels of electronic measurement instruments, by the camera embedded on the drone. Furthermore, we allow remote student-teacher communication using the drone, in case there is a teacher present in the remote lab facility. Finally, the drone has to go back home when the mission is over to land on a platform for automatic recharge of the batteries. Second, we propose an automatic system that estimates the affective state of the student (frustrated/confused/flow) in order to take appropriate interventions to ensure good learning outcomes. For example, if the studen is having major difficulties we can try to give him hints or to reduce the difficulty level of the lab experiment. We propose to do this by using visual cues (head pose estimation and facil expression analysis). Many evidences on the state of the student can be acquired, however these evidences are incomplete, sometims inaccurate, and do not cover all the aspects of the state of the student alone. This is why we propose to fuse evidences using the theory of Dempster-Shafer that allows the fusion of incomplete evidence

Enrichir les laboratoires distants grâce à la vision par ordinateur avec drone.

Avec le progrès technologique, de nouvelles technologies sont en cours de développement afin de contribuer à une meilleure expérience dans le domaine de l’éducation. En particulier, les laboratoires distants constituent un moyen intéressant et pratique qui peut motiver les étudiants à apprendre. L'étudiant peut à tout moment, et de n'importe quel endroit, accéder au laboratoire distant et faire son TP (travail pratique). Malgré les nombreux avantages, les technologies à distance dans l’éducation créent une distance entre l’étudiant et l’enseignant. Les élèves peuvent avoir des difficultés à faire le TP si aucune intervention appropriée ne peut être prise pour les aider. Dans cette thèse, nous visons à enrichir un laboratoire électronique distant conçu pour les étudiants en ingénierie et appelé «LaboREM» (pour remote laboratory) de deux manières: tout d'abord, nous permettons à l'étudiant d'envoyer des commandes de haut niveau à un mini-drone disponible dans le laboratoire distant. L'objectif est d'examiner les faces-avant des instruments de mesure électroniques, à l'aide de la caméra intégrée au drone. De plus, nous autorisons la communication élève-enseignant à distance à l'aide du drone, au cas où un enseignant serait présent dans le laboratoire distant. Enfin, le drone doit revenir pour atterrir sur la plate-forme de recharge automatique des batteries, quand la mission est terminée. Nous proposons aussi un système automatique pour estimer l'état de l'étudiant (frustré / concentré..) afin de prendre les interventions appropriées pour assurer un bon déroulement du TP distant. Par exemple, si l'élève a des difficultés majeures, nous pouvons lui donner des indications ou réduire le niveau de difficulté de l’exercice. Nous proposons de faire cela en utilisant des signes visuels (estimation de la pose de la tête et analyse de l'expression faciale). De nombreuses évidences sur l'état de l'étudiant peuvent être acquises, mais elles sont incomplètes, parfois inexactes et ne couvrent pas tous les aspects de l'état de l'étudiant. C'est pourquoi nous proposons dans cette thèse de fusionner les preuves en utilisant la théorie de Dempster-Shafer qui permet la fusion de preuves incomplètes.With the technological advance, new learning technologies are being developed in order to contribute to better learning experience. In particular, remote labs constitute an interesting and a practical way that can motivate nowadays students to learn. The student can at any time, and from anywhere, access the remote lab and do his lab-work. Despite many advantages, remote technologies in education create a distance between the student and the teacher. Without the presence of a teacher, students can have difficulties, if no appropriate interventions can be taken to help them. In this thesis, we aim to enrich an existing remote electronic lab made for engineering students called “LaboREM” (for remote Laboratory) in two ways: first we enable the student to send high level commands to a mini-drone available in the remote lab facility. The objective is to examine the front panels of electronic measurement instruments, by the camera embedded on the drone. Furthermore, we allow remote student-teacher communication using the drone, in case there is a teacher present in the remote lab facility. Finally, the drone has to go back home when the mission is over to land on a platform for automatic recharge of the batteries. Second, we propose an automatic system that estimates the affective state of the student (frustrated/ confused/ flow..) in order to take appropriate interventions to ensure good learning outcomes. For example, if the student is having major difficulties we can try to give him hints or reduce the difficulty level. We propose to do this by using visual cues (head pose estimation and facial expression analysis). Many evidences on the state of the student can be acquired, however these evidences are incomplete, sometimes inaccurate, and do not cover all the aspects of the state of the student alone. This is why we propose to fuse evidences using the theory of Dempster-Shafer that allows the fusion of incomplete evidence

Visual localization and servoing for drone use in indoor remote laboratory environment

International audienceIn this paper, we present a localization system for the use of drone in a remote lab. The objective is to allow a drone to inspect remote electronic instruments autonomously, as well as to return to its base and land on a platform for the recharge of its batteries. In addition, the drone should be able to detect the presence of a teacher in the lab, and to center the human face in the image in order to enable remote student-teacher communication. To achieve the first objective, the localization approach is composed of a monocular SLAM (Simultaneous Localization and Mapping) algorithm PTAM (Parallel Tracking and Mapping) and an estimation based on the homography transform. For the face-drone servoing, the approach is based on the 3D Candide model. Both approaches work in real-time. Quantitative and qualitative experiments are presented that show the robustness of both methods

USING COMPUTER VISION FOR STUDENT-CENTRED REMOTE LAB IN ELECTRONICS

WOS:000402955900080International audienceTechnology is affecting students' life in different ways. Nowadays, students are more interested in social media, games and Internet of Things. On the other hand, most current teaching techniques appear outdated and uninteresting to students due to the fact that they do not follow the technological advances. For these reasons, students nowadays do not show much interest in education and learning. On the other hand, there is an urgent need for new learning approaches that take into account both the interests of today's students and the technological advances in several fields like telecommunication and gaming. Particularly for STEM curricula (Science, Technology, Engineering, and Math), labs play a crucial role to help students fully understand the material given during lectures. However, traditional hands-on labs are tied to space-time constraints: a student must perform the lab activity in a pre-scheduled time and in a given space. Furthermore, traditional labs are not suited for all students: some students may need more help and more motivating factors in order to take the most of the lab experience and to enhance their learning outcomes. This is not always best done in hands-on labs due to large number of students and to the presence of only one teacher that has to do lots of effort to adapt to each student's needs. Remote labs provide an alternative experience to the traditional lab, eliminating time-space constraints and introducing students to a new way of working by manipulating hardware remotely, which is used frequently in industry (SCADA: Supervisory Control and Data Acquisition). It can also be adapted to each student's needs if one is capable to infer student's mood, emotion and competence. In this paper we present the implementation of a remote lab in electronics, called LaboREM, which takes advantage of the technological advances in telecommunication and takes into consideration what current students are interested in. It is based on three parts: a remote laboratory, a learning management system and a game-like approach. The remote laboratory consists of remotely-controlled measurement devices, plus a simple robotic arm that mimics the student hand to construct electronic circuits. The robotic arm picks up electronics components equipped with magnets and places them on a breadboard. A wide angle camera is installed in order that students see what is physically happening in the lab. The electronic lab activity consists of short-time experiments (mainly tests on active filters) that last less than 5 minutes. The scenario is based on a game-like approach: a treasure hunt and a Top 10. Furthermore, a mini drone that can be controlled by the student remotely complements the static camera to add more immersion to the lab experience. The student can send specific requests to the drone in order to see some electronic instruments in a more interesting way. Moreover image processing and pattern recognition techniques are also used to infer student mood, concentration and motivation. Clues of boredom, lack of motivation and concentration (closed eyes, student not looking at the screen, student keeping moving…) are detected from the video of students' face taken from a webcam in order to adapt the scenario and the lab to each student